1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method, and specifically relates to an image processing apparatus and an image processing method, which detect movements of a subject appearing in a plurality of images.
2. Description of the Related Art
As a method for detecting movements of a subject appearing a plurality of images, the technique of estimating where a feature point (pixel) in an image that serves as a reference (hereinafter, referred to as “reference image”) have moved in another image (hereinafter, referred to as “tracking image”), that is, the technique of tracking (estimating) the locus (optical flow) of the feature point, has been proposed.
Gradient methods, which are typified by the Lucas-Kanade method, are representatives of the technique of tracking a feature point through a plurality of images. In a gradient method, a displacement (movement amount) of a feature point is figured out from the gradient (first-order derivative) of an image signal in each of the temporal direction and the spatial direction. For example, as described in Japanese Patent No. 3435084, a displacement of a feature point is calculated by solving optical flow, temporal and spatial derivative constraint equations under predetermined conditions.
In general, gradient methods have a problem in that the accuracy of calculation of displacement of a subject is lowered when the subject makes a large movement. As methods for solving such problem, hierarchical gradient methods have been proposed (Japanese Patent No. 3435084 and Japanese Patent Application Laid-Open No. 2007-49545). In a hierarchical gradient method, inverted pyramid hierarchical tier image groups are created by means of stepwise reduction of a reference image and a tracking image to form inverted pyramid hierarchies, the coordinates of a feature point are detected in the hierarchical tier images in the inverted pyramid hierarchical tier image groups in increasing order of image size (pixel count) (i.e, in increasing order of feature point displacement amount (pixel count) upon movement), and when detecting the feature point in a hierarchical tier image with a larger image size, the detection result (tracking result) of the feature point in a hierarchical tier image with a smaller image size is reflected.
In gradient methods, a predetermined feature point in a reference image can be tracked more correctly by repeatedly detecting a point corresponding to the feature point in a tracking image. More specifically, the position of a feature point (tracking point) in a tracking image is repeatedly figured out. Then, when the difference value (displacement amount) between the coordinate values of the latest tracking point and the coordinate values of the tracking point obtained immediately before the latest tracking point is less than a predetermined value, the displacement value is determined as having converged and the repetitive tracking point detection is terminated. This process is repeated for each hierarchical tier image, enabling more correct tracking of the feature point.
However, the easiness of convergence of a tracking point displacement amount varies depending on the image patterns around the feature point and the tracking point (for example, image signal (brightness) distribution). In a hierarchical gradient method, a tracking point is repeatedly detected for each hierarchical tier of an inverted pyramid hierarchical tier image group, and thus, use of the aforementioned method incurs the waste of excessive repetition of the tracking point detection process and the displacement calculation process, resulting in inefficiency.